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Abstract. Probabilistic timed automata (PTAs) are used for formal 
modelling and verification of systems with probabilistic, nondeterminis- 
tic and real-time behaviour. For non-probabilistic timed automata, for- 
wards reachability is the analysis method of choice, since it can be im- 
plemented extremely efficiently. However, for PTAs, such techniques are 
only able to compute upper bounds on maximum reachability probabil- 
ities. In this paper, we propose a new approach to the analysis of PTAs 
using abstraction and stochastic games. We show how efficient forwards 
reachability techniques can be extended to yield both lower and up- 
per bounds on maximum (and minimum) reachability probabilities. We 
also present abstraction-refinement techniques that are guaranteed to 
improve the precision of these probability bounds, providing a fully au- 
tomatic method for computing the exact values. We have implemented 
these techniques and applied them to a set of large case studies. We 
show that, in comparison to alternative approaches to verifying PTAs, 
such as backwards reachability and digital clocks, our techniques exhibit 
superior performance and scalability. 


1 Introduction 


Probabilistic behaviour occurs naturally in many real-time systems, either due 
to the use of randomisation, or because of the presence of unreliable components. 
Prominent examples include communication protocols such as Bluetooth, IEEE 
802.11 and FireWire, which use randomised back-off schemes and are designed 
to function over faulty communication channels. Another important class are 
security protocols, such as for non-repudiation, anonymity and non-interference, 
where randomisation and timing are both essential ingredients. 

Probabilistic timed automata (PTAs) [9[1J16], which are finite state automata 
extended with real-valued clocks and discrete probabilistic choice, are a natural 
formalism for modelling and analysing such systems. Formal verification tech- 
niques for PTAs can help to identify anomalies resulting from the subtle interplay 
between probabilistic, real-time and nondeterministic aspects of these systems. 
A fundamental property of a PTA is the minimum or maximum probability of 
reaching a particular class of states in the model. This allows the expression of 
a wide range of useful properties, for example, “the minimum probability that a 
data packet is correctly delivered with t seconds”. 
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There are three main existing algorithmic approaches to the verification of 
PTAs: (i) forwards reachability [I6[5]; (ii) backwards reachability [I7]; and (iii) 
digital clocks [I5]. Forwards reachability is based on a symbolic forwards explo- 
ration, similar to the techniques implemented in state-of-the art tools for non- 
probabilistic timed automata [618]. This approach is appealing because it can be 
implemented extremely efficiently with data structures such as difference-bound 
matrices (DBMs). However, in the context of probabilistic timed automata, these 
techniques yield only an upper bound on maximum reachability probabilities. 

Backwards reachability [I7] performs a state-space exploration in the oppo- 
site direction, from target to initial states. This computes exact values for both 
minimum and maximum reachability probabilities; however, the operations re- 
quired to implement it are expensive, limiting its applicability. The digital clocks 
technique of [I5] uses an efficient language-level translation to a probabilistic 
model with finite state semantics. This also gives precise values for minimum 
and maximum probabilities, but is only applicable to a restricted class of PTAs. 

PTAs are, because of their real-valued model of time, inherently infinite-state. 
The three PTA verification techniques described above work by constructing a 
finite-state Markov decision process (MDP) that can be analysed with existing 
tools and techniques. This MDP can be viewed as an abstraction of the infinite- 
state semantics of the PTA. In this paper, we take a new approach, using the 
ideas of [I3] to represent PTA abstractions as stochastic two-player games. 

We first show how the forwards reachability technique of can be gen- 
eralised to produce a stochastic game that yields lower and upper bounds on 
either minimum or maximum reachability probabilities of PTAs. Then, using 
abstraction-refinement methods, we show how the stochastic game can be itera- 
tively refined in order to tighten these bounds. This gives a fully automatic tech- 
nique to compute exact reachability probabilities within a finite number of steps. 
Finally, we present a prototype tool implementing these techniques that exhibits 
significantly better performance than other PTA verification approaches. A full 
version of this paper, including proofs is also available [14]. 


Related work. Existing PTA verification techniques are discussed above and 
a detailed experimental comparison is included in Section [6] Also relevant is 
[3], which presents an algorithm for computing time-abstracting bisimulation 
quotients of PTAs. Abstraction-refinement approaches have been proposed for 
non-probabilistic timed automata, e.g. which uses bounded model checking 
and SAT-based techniques, [2I] which is based on the region graph construction, 
and [7] for verifying PLC automata using UPPAAL [I8]. 


2 Markov decision processes and stochastic games 


Markov decision processes (MDPs) are a widely used formalism for modelling 
systems that exhibit both nondeterministic and probabilistic behaviour. 
Definition 1. An MDP M is a tuple (S, S, Act, Steps) where S is a set of 
states, S C S is the set of initial states, Act is a set of actions and Stepsy : 
Sx Act — Dist(S) is the probabilistic transition function. 
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In each state s € S of an MDP M, there is a nondeterministic choice between 
one or more available actions a € Act (those for which Stepsy(s,a) is defined). 
After the choice of an action a, a successor state is selected at random according 
to the probability distribution Stepsm(s,a). A path through M is a sequence of 
states selected in this fashion. 

To reason about the MDP M, we use the notion of an adversary, which is a 
possible resolution of all nondeterministic choices in M (formally, an adversary 
is a function from finite paths to actions). For a fixed adversary A, we can define 
a probability measure over the set of paths from a state s and, in particular, the 
probability p4(F) of reaching a target FCS from s under A. We are typically 
interested in the minimum and maximum reachability probabilities for F: 


pM (F) = inf,eg infa på (F) and py*(F) = supseg supa pe (F). 


These values, and an adversary of M which produces them, can be computed 
with a simple numerical computation called value iteration [I9]. 


Stochastic two-player games [P04] extend MDPs by allowing two types of 
nondeterministic choice, controlled by separate players. We use stochastic games 
in the manner proposed in [13] to represent an abstraction of an MDP. 


Definition 2. A stochastic game G is a tuple (9,9, Act, Stepsg) where: S is 
a set of states, S C S is the set of initial states Act is a set of actions and 
Steps. : Sx Act > 2Pist(S) is the probabilistic transition function. 


Each transition of a stochastic game G comprises three choices: first, like for an 
MDP, player 1 picks an available action a€ Act; next, player 2 selects a distri- 
bution A from the set Steps¢(s, a); finally, a successor state is chosen at random 
according to À. A resolution of the nondeterminism in G (the analogue of an 
MDP adversary) is a pair of strategies 01, 02 for the players, under which we can 
define the probability p397? (F) of reaching a target FCS from a state s. 

Intuitively, the idea of [I3] is that, in a stochastic game G, representing an 
abstraction of an MDP M, player 2 choices represent nondeterminism present in 
M and player 1 choices represent additional nondeterminism introduced through 
abstraction. By quantifying over strategies for players 1 and 2, we can obtain 
both lower bounds (lb) and upper bounds (ub) on the minimum and maximum 
reachability probabilities of M. If G is constructed from M using the approach 
of [I3], then, in the case of maximum probabilities, for example: 


pe (F) < pi (F) < pe (F) 
where, in the stochastic game G: 


pe “(F) = sup,eg inf, sup,, poh? (F) 
b, def io 
De mes (F) = SUP 6g SuPs, SUP,, py (F) 
Using similar techniques as those for MDPs, we can efficiently compute these 
values and strategies for players 1 and 2 that result in them [4]. 


4 Marta Kwiatkowska, Gethin Norman, David Parker 


3 Probabilistic Timed Automata 


Time, clocks and zones. Probabilistic timed automata model time using 
clocks, variables over the set R of non-negative reals. We assume a finite set 
X of clocks. A function v : ¥ —> R is referred to as a clock valuation and the set 
of all clock valuations is denoted by R*. For any v € RY, t € R and X C X, we 
use v+t to denote the clock valuation which increments all clock values in v by 
t and v[X:=0] for the valuation in which clocks in X are reset to 0. 

The set of zones of X, written Zones(4), is defined by the syntax: 


¢u=true|a<d|c<a|ate< ytd|ac|¢cv¢ 


where x,y € ¥ and c,d € N. A zone ¢ represents the set of clock valuations 
v which satisfy Ç, denoted v < Ç, i.e. those for which Ç resolves to true by 
substituting each clock x with v(x). 

We will use several classical operations on zones [8]22]. The zone Z¢ contains 
all clock valuations that can be reached from a valuation in ¢ by letting time 
pass. Conversely, ,“¢ contains those that can reach ¢ by letting time pass. For 
XCX, the zone [X:=0]¢ contains the clock valuations which result in a valuation 
in Ç when the clocks in X are reset to 0, while ¢[X:=0] contains the valuations 
obtained from those in ¢ by resetting these clocks to 0. 


Syntax and semantics of PTAs. We now present the formal syntax and 
semantics of probabilistic timed automata. 


Definition 3. A PTA is a tuple P=(L,1, Act, inv, enab, prob) where: 


— L is a finite set of locations and T € L is the initial location; 

— Act is a finite set of actions; 

— inv: L > Zones(&) is the invariant condition; 

— enab: Lx Act > Zones(&) is the enabling condition; 

— prob : Lx Act — Dist(2* x L) is the probabilistic transition function. 


A state of a PTA is a pair (l, v) € LxR* such that v<inv(l). In any state (l, v), 
a certain amount of time t € R can elapse, after which an action a € Act is 
performed. The choice of t requires that, while time passes, the invariant inv(l) 
remains continuously satisfied. Each action a can be only chosen if it is enabled, 
that is, the zone enab(l,a) is satisfied by v+t. Once action a is chosen, a set 
of clocks to reset and successor location are selected at random, according to 
the distribution prob(l, a). We call each element (X,/') € 2¥ xL in the support 
of prob(l,a) an edge and, for convenience, assume that the set of such edges, 
denoted edges(l, a), is an ordered list (e1,...,€n). 


Definition 4. Let P=(L,1, Act, inv, enab, prob) be a PTA. The semantics of P 
is defined as the (infinite-state) MDP [P] = (S,S,Rx Act, Stepsp) where: 


— S={(l,v) € Lx RY | vsino(l)} and S = {(1,0)}; 
— Stepsp((l, v), (t,a)) =A if and only if v+t <inv(l) for all 0<t'<t, 
v+t < enab(l,a) and, for any (l’,v') € S: 


AE 2 l= s {| prob(l,a)(X,1') | X € 2% Av’ = (v4+t)[X:=0] h ; 
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Each transition of the semantics of the PTA is a time-action pair (t, a), represent- 
ing time passing for t time units, followed by a discrete a-labelled transition. If 
Stepsp((l, v), (t,a)) is defined and edges(l,a) = ((l,,.X1),---; (In, Xn)), we write 
(1, v) =, (51,..-, Sn) where s; = (l;, (v+t)[X;:=0]) for alll Si <n. 

We make several standard assumptions about probabilistic timed automata. 
Firstly, we restrict our attention to structurally non-Zeno automata [23]. This 
class of models, which can be identified syntactically and in a compositional fash- 
ion , guarantees time-divergent behaviour. Secondly, for technical reasons, we 
assume all zones appearing in a PTA are diagonal-free [2]. 


Probabilistic Reachability. The minimum and maximum probabilities of 
reaching, from the initial state of a PTA P, a certain target F C L are: 


pR™(F) = pppp(Se) and pR™(F) = pi (Sr) 


where Sp = {(l,v)|u<inu(l) Al € F}. We can easily consider more expressive 
targets, that refer to both locations and clock values, through a simple syntactic 
modification of the PTA [16]. 


Symbolic states and operations. In order to represent sets of PTA states, 
we use the concept of a symbolic state: a pair z = (l, C), comprising a location | 
and a zone Ç over Æ, representing the set of PTA states {(1,v)|v<¢}. We use 
the notation (l, v) € (l, Ç) to denote inclusion of a PTA state in a symbolic state. 

We will use the time successor and discrete successor operations of [S]22]. 
For a symbolic state (l, ¢), action a, and edge e = (X,l’) € edges(l, a), we define: 


— tsuc(1,¢) = (L inv(1)A ZC) is the time successor of (L, ); 
— dsuc[a, e](1,¢) = (V, (CAenab(I, a))[X:=0]Ainv(I’)) is the discrete successor 
of (lL, ¢) with respect to e; 


— posta, e](1,¢) = tsuc(dsuc{a, e](1,¢)) is the post of (1,¢) with respect to e. 


The c-closure of a zone ¢ is obtained by removing any constraint that refers to 
integers greater than c. For a given c, there are only a finite number of c-closed 
zones. For the remainder of this paper, we assume that all zones are c-closed 
where c is the largest constant appearing in the PTA under study. 


4 Forwards Reachability for PTAs 


In this section, we begin by describing the approach of [I6], which we will refer 
to as MDP-based forwards reachability. This computes only upper bounds on 
maximum reachability probabilities of a PTA. Subsequently, we will propose a 
new algorithm, based on stochastic games, which addresses these limitations. 


4.1 MDP-based forwards reachability 


The MDP-based forwards reachability approach of [I6] works by building an 
abstraction of a PTA P. This abstraction is represented by an MDP M whose 
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BuildReachGraph(P, F`) 


Z:=0 
Y := {tsuc(1, 0)} 
while Y 4 0) 
choose (1,¢) €Y 
Y= Y\ {0} 
z= ZU {0,0} 
for a € Act such that enab(l,a) \¢ #0 
for e; € edges(l,a) = (e1,..., en) 
(Ui, i) == post[(I, a), es] (1, €) 
if (1;,¢;) Z Z and l; ¢ F then Y := YU {(lj, G})} 
11 R := RU {((L Ç), a, (l1; 01), +- +» (Ins Ca)))} 
12 return (Z,R) 


o Noan AUOUNe 


j= 
oo 


BuildMDP(z, R) 


zengen 

for (J,¢) € Z and 0 € R(I, ¢) 
Stepsy((l, C), 0) := Ao 

return M = (Z,Z,R, Steps) 


Ae Ne 


Fig. 1. Algorithm for MDP-based forwards reachability, based on [16] 


state space is a set Z of symbolic states, i.e. each state of M represents a set of 
states of the infinite-state MDP semantics [P]. The algorithm of is shown 
in Figure |1| For the purposes of this presentation, we have reformulated the 
algorithm into: (i) the construction of a reachability graph over the set of symbolic 
states Z; and (ii) the construction of an MDP M from this graph. 

The algorithm to build this reachability graph is based on the well-known 
forwards reachability algorithm for non-probabilistic timed automata [618]. It 
performs a forwards exploration through the automata, successively computing 
symbolic states using the post operation. One important difference is that, in 
the probabilistic setting, on-the-fly techniques cannot be used: the state-space 
exploration is exhaustive. This is because the aim is to determine, not just the 
existence of a path to the target, but the probability of reaching the target. For 
this, an MDP containing all such paths is constructed and analysed. 

A reachability graph captures information about the transitions in a PTA. It 
comprises a multiself| z of symbolic states and a set R C Zx ActxZ* of symbolic 
transitions. Each symbolic transition 0 € R takes the form: 


0 = ((L, Ç), a, (l, &), a (lisGa))) 


where n = |edges(l,a)|. Intuitively, 0 represents the possibility of taking action 
a from a PTA state in (l, Ç) and, for each edge (X;,1;) € edges(l, a), reaching a 
state in (l;, Çi). A key property of symbolic transitions is the notion of validity: 


valid(0) = CA / (enab(l, a)A (AT, ([Xi:=0]G)) ) 


1 The use of a multiset is a technical requirement, later used for abstraction refinement. 
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which gives precisely the set of clock valuations satisfying ¢ from which it is 
possible to let time pass and perform the action a such that taking the ith edge 
(Xi, li) gives a state in (l, ¢;). A symbolic transition @ is valid if the zone valid (0) 
is non-empty. This leads to the following formal definition of a reachability graph. 


Definition 5. A reachability graph for a PTA P=(L, 1, Act, inv, enab, prob) and 
target F, is a pair (Z,R) where: 


— ZC LxZones(&) is a multiset of symbolic states where {s € z|z € Z} = S; 
— RC ZxActxZ* is a set of valid symbolic transitions; 


and, ifz = (1,¢)€Z,1¢F,s€zands = (S1,---,8n), then R contains a 
symbolic transition (z, a, (Z1,..-,Zn)) such that si € z; for alll <i<n. 


For any PTA P and target F, it follows from the definition of post that algo- 
rithm BuildReachGraph(P, F) in Figure [I| returns a (unique) reachability graph 
for (P, F). However, the above conditions do not imply the uniqueness of reach- 
ability graphs, and there may exist many other such graphs for (P, F). 

Given a reachability graph (Z,R) we can construct an MDP M with state 
space Z using the symbolic transitions in R to build the transitions of M. More 
precisely, a symbolic transition 0 = ((1,¢),a, ((l1,¢1),---, (In, Gn))) induces a 
probability distribution Ag over symbolic states Z where for any (l’, ¢’) € Z: 


Aol, C’) E © {| prob(1, a)(ex) |e; € edges(l,a) A G=C' I} . 


Using these distributions, the algorithm BuildMDP(Z,R) in Figure [1] constructs 
an MDP M, analysis of which yields bounds on the behaviour of P. 


Theorem 1. Let P be a PTA with target F. If (Z,R) is a reachability graph 
for (P, F) and M is the MDP returned by BuildMDP(Z,R) (see Figure Hp, then 
pE (Zr) <p (F) and pS**(F) < pm®*(Zr) where Zp = Fx Zones(X). 


This theorem extends [I6], by establishing the result for any reachability graph, 
not just that returned by BuildReachGraph and, by restricting to structurally 
non-Zeno PTAs, also yields lower bounds on minimum reachability probabilities. 


Example 1. We illustrate these ideas using the simple PTA P in Figure [2{a). 
We use the standard graphical notation for PTAs and omit probability 1 labels. 
Applying BuildReachGraph(P, {l3}) (see Figure [1} yields the symbolic states: 


Z= {(lo, r=y), (hi, r=y), (i, y<a—2), (le, IXY), (Is, r=y)} 


and the set of symbolic transitions R. From the first two symbolic states, for 
example, we have R(lo, s=y) = {0a} and R(l1, c=y) = {05, 0e} where: 


Oa = (lo, x=y), a, (h, r=y), (l2, £z<y))) 
I, = (l1,2=y), b, ((4,¢=y))), be = ((1,2=y), C, (ls, «=y))) 
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a=, (l3,v=y) (li, y<a—2) 
uE o 


(h,z=y) (l2, £y) 


0.6 o 0.4 0.6 0.4 
ge 
(a) PTA (b) MDP 


Fig. 2. Analysis of a PTA through MDP-based forwards reachability 
BuildGame(Z, R) 


Stepsc((l, €), O) := {ào | 0 € O} 
return G = (Z, Z, 2", Steps.) 


EAER 

2 for (1,¢) €Z 

3 for O CR(I,¢) such that O 4 ý and valid(O) 
4 

5 


Fig. 3. Algorithm to construct a stochastic game from a reachability graph 


The resulting MDP is shown in Figure[2(b)| The maximum probability of reach- 
ing location /3 in the PTA is 0.6, which results from taking action a in lọ imme- 
diately and, if lı is reached, proceeding straight to l3. An alternative is to wait 
for 1 time unit in lọ and then take a, reaching l3 via l2, however, this results 
in a lower probability of 0.4. An upper bound on the maximum probability for 
the PTA is obtained from the maximum probability of reaching (/3,2=y) in the 
MDP. The resulting value is 1. This is because the symbolic states for locations 
lı and l2 are too coarse to preserve the precise time that action a is taken. 


4.2 Game-based forwards reachability 


The main limitation of the MDP-based forwards reachability algorithm is that 
it only provides lower bounds for minimum and upper bounds for maximum 
reachability probabilities. We now describe how to construct, from a reachability 
graph, a stochastic game G that yields both lower and upper bounds. The game 
G is, like the MDP in the previous section, an abstraction of the infinite-state 
MDP semantics of the PTA, whose state space is the symbolic states Z. 

We utilise the approach of [I3] to represent an abstraction of an MDP as a 
stochastic two-player game. The basic idea is that the two players in the game 
represent nondeterminism introduced by the abstraction and nondeterminism 
from the original model. In a symbolic state (l, Ç) of the game abstraction of a 
PTA, player 1 first picks a PTA state (/,v) € (1,¢) and then player 2 makes a 
choice over the actions that become enabled after letting time pass from (l, v). 

In order to construct such a game from a reachability graph (Z,R), we first 
extend the notion of validity to sets of symbolic transitions with the same source. 
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(ls,a=y) (h,y<a— a (l3, x=y) (li, y<a—2 
(l, c= =y) (Ig, v<y) (11, c=y=0) (4, x= y>0) (l2, £y) 
0.4 
0.6 0.4 os 0.6 0.4 
(lo, r=y) (lo, r=y) 
4 A 
(a) From reachability graph (b) After one refinement 


Fig. 4. Stochastic games for the PTA example of Figure [2] 
For any symbolic state (l, Ç) € Z and set of symbolic transitions © C R(I, Ç), let: 


valid(®) = (Agcovalid(0)) A (Agenci,c)\e7valid()) . 


By construction, valid(@) identifies precisely the clock valuations v < ¢ such 
that, from (l, v), it is possible to perform a transition encoded by any symbolic 
transition 0 € O, but it is not possible to perform a transition encoded by any 
other symbolic transition of R(l, Ç). 

The algorithm BuildGame in Figure |3| describes how to construct, from a 
reachability graph R, a stochastic game with symbolic states Z. In a state z 
of the game, player 1 chooses between any non-empty valid set of symbolic 
transitions © C R(z). Player 2 then selects a symbolic transition 0 € O. As the 
following result demonstrates, this game yields lower and upper bounds on either 
minimum or maximum reachability probabilities of the PTA. 


Theorem 2. Let P be a PTA with target F. If (Z,R) is a reachability graph for 
(P, rI mi G is the stochastic game returned by BuildGame(Z,R) (see Figure[3), 
then pe *(Zr) < pp(F) < pe” *“(Zr) for x € {min, max}. 


Example 2. We return to the PTA from Figure |2| and the reachability graph 
constructed in Example [I] The corresponding stochastic game is shown in Fig- 
ure As for PTAs and MDPs, we draw probability distributions as arrows 
grouped by an arc, omitting the labelling of probability 1 transitions. A set 
of distributions emanating from a black circle indicates a player 2 choice; the 
outgoing edges from each symbolic state represent a player 1 choice. 

Consider, the symbolic state (l1, =y), for which there are two symbolic tran- 
sitions 6, and ĝe (see Example[I). Since valid(0))=(a=y) and valid(@,.)=(x=y=0), 
we have valid({6,})=(x=y>0), valid({O.})=0 and valid({4p, 0e })=(x=y=0). This 
tells us that there are two classes of PTA states in (l1, «=y): those in which both 
actions b and c become enabled, and those in which only b becomes enabled. 
Thus, in the game state (see Figure [4(a)), we see that player 1 chooses between 
these two classes and then player 2 chooses an available action. 

Using Theorem [2| the stochastic game in Figure zo gives bounds on the 
maximum probability of reaching l3 in the PTA. The upper bound (as for the 
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Refine(Z, R, (l, C), Ow, Our) 


1 Cp = valid (Ow) 

2 Cup := valid(Ou) 

3 zee = {6 Çe), (L ah (L, CA (i Vus) )} \ {10} 
4 ZI NGOer 

5 RY:=0 

6 for 6 = (z0,a,(Z1,...,Zn)) ER 

7 ~~ if (l, Ç) Z {Zz0,Z1,..., Zn} then 

8 RS := RFU {0} 

9 else 
10 ore’ := {(z0,a, (z1,-.-,Z2n)) | z; € 2” if z; = (l, Ç) and z; = zi o/wise} 
11 for 0” € O"™ such that valid (0”®®) # Ø 


12 RÍ — RU {grew} 
13 return (Z°, R") 


Fig. 5. Algorithm to refine symbolic state (l, Ç) in reachability graph (Z, R) 


MDP) is 1 as, after either branch of the initial probabilistic choice, player 1 can 
make a choice which allows l3 to be reached with probability 1. The lower bound, 
however, is 0 because player 1 can also, in both cases, make l unreachable. 


As the above example illustrates, it is possible that the difference between the 
lower and upper bounds from the game is too great to provide useful information. 
In the next section, we will address this issue by introducing a way to refine the 
abstraction to reduce the difference between the bounds. 


5 Abstraction Refinement 


The game-based abstraction approach of [I3] has been extended with refinement 
techniques in [IONI]. Inspired by non-probabilistic counterexample-guided ab- 
straction refinement, the idea is that an initially coarse abstraction is iteratively 
refined until it is precise enough to yield useful verification results. Crucial to 
this approach is the use of the lower and upper bounds provided by a stochastic 
game abstraction as a quantitative measure of the preciseness of the abstraction. 


The refinement algorithm. Our refinement algorithm takes a reachability 
graph (Z,R), splits one or more of the symbolic states in Z and then modifies the 
symbolic transitions of R accordingly. This process is guided by the analysis of 
the stochastic game constructed from (Z,R), i.e. the bounds for the probability 
of reaching the target and player 1 strategies that attain these bounds. 

We now outline the refinement of a single symbolic state (/,¢) for which 
the bounds differ and for which distinct player 1 strategies yield each bound? 
A player 1 strategy chooses, for any state in the stochastic game, an action 
available in the state. By construction, an available action in (l, Ç) is a valid set 


? From the results of such a state exists when the bounds differ in some state. 
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AbstractRefine(P, F, x, €) 


(Z,R) := BuildReachGraph(P, F`) 
G := BuildGame(Z, R) 
(ve, pe?*, ot, ot”) := AnalyseGame(G, F, +) 
while pt?*—pe* >e 
choose (1,¢) €Z 
(Z,R) := Refine(Z,R, (l, ¢), o? (I, €), of (1, 0) 
G := BuildGame(Z, R) 
(ve*, pe’*, of? ot”) := AnalyseGame(G, F, x) 


return [pe”, al 


CANA KwWNH 


Fig. 6. Abstraction-refinement loop to compute reachability probabilities 


of symbolic transitions from R(l, ¢). We let Om, Ou» C R(L, Ç) denote the distinct 
player 1 strategy choices for the lower and upper bound respectively. Since the 
validity conditions for Om and Ou» identify precisely the clock valuations in ¢ 
for which the corresponding transitions of [P] are possible, we split (J,¢) into: 


(1, valid(Oy)), (1, valid(Ou»)) and (1,¢ A a(valid(Oy,) V valid(Ou,))) . 


By construction, valid(O,,) and valid(©,,») are both non-empty. Furthermore, 
since Oj, # Oxy, from the definition of validity, we have valid(@) A valid(O’) = 9, 
and hence the split of (l, Ç) produces a strict refinement of Z. 

The complete refinement algorithm is shown in Figure [5] Lines 1-4 refine Z, 
as just described, and lines 5-12 update the set of symbolic transitions R. The 
result is a new reachability graph, for which the corresponding stochastic game 
is a refined abstraction of the PTA, satisfying the following properties. 


Theorem 3. Let P be a PTA with target F and (Z,R) be a reachability graph 
for (P, F). If (2, RÊ) is the result of applying algorithm Refine (see Pigure(a) 
to (Z,R), G = BuildGame(Z,R) and G"*! = BuildGame(Z",R"?), then: 

(i) (Z$, RT!) is a reachability graph for (P, F); 

(it) pe* (Zr) < per (Zr) and pers (Zr) < pe* (Zr) for x € {min, max}. 
This refinement scheme, applied in a iterative manner, provides a way of comput- 
ing exact values for minimum or maximum reachability probabilities of a PTA. 
This algorithm, outlined in Figure a starts with the reachability graph con- 
structed through forwards reachability and then repeatedly: (i) builds a stochas- 
tic game; (ii) solves the game to obtain lower and upper bounds; and (iii) refines 
the reachability graph, based on an analysis of the game. The iterative process 
terminates when the difference between the bounds falls below a given level of 
precision €. In fact, as the following result states, this process is guaranteed to 
terminate, in a finite number of steps, with the precise answer. 


Theorem 4. Let P be a PTA with target F and x € {min, max}. The algorithm 
AbstractRefine(P, F,x,0) (see Figure[6) terminates after a finite number of steps 


lb,x ub,» lb,x m ub,» 


and returns [po , pe] where pg” = pp (F) = pe 
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Example 3. We return to our running example (see Figures [2] and [4) and con- 
sider the refinement of (/,,2=y), from which the lower and upper bounds on the 
maximum probability of reaching location l3 are 0 and 1. The player 1 strategies 
(see Example [2) to achieve these bounds select Oj, = {6,} and Ouo = {05, 0c}, 
respectively. The validity conditions for these choices are (x=y>0) and (x=y=0), 
and hence (l1, «=y) is divided into zı = (l1,a=y>0) and zg = (l1, v=y). 

We then update the set R, as described in Figure [5] splitting symbolic tran- 
sitions whose source or target is (l4, x=y). For example, ĝa, 0p and 6, (see Ex- 
ample [I) are split into, for i = 1,2: 


6° =((Io, x=y), a, (zi, (l2, r<y))), O=(zi, b, (zi)) and o=(zi, c, ((l3, x=y=0))). 


After removing 62, which is not valid, the resulting stochastic game is shown 


in Figure |4(b)| While this still yields bounds of [0,1] for the initial state, two 
subsequent refinement tighten this to [0.6, 1.0] and then [0.6, 0.6]. 


6 Experimental Results 


Implementation. We have implemented a prototype PTA model checker based 
on the techniques in this paper. It uses difference-bound matrices (DBMs) to rep- 
resent zones. Since refinement can introduce non-convex zones, we also employ 
lists of DBMs. Our tool takes a textual description of a PTA (or the parallel 
composition of several PTAs) and a set of target locations. It then executes 
the abstraction-refinement loop described in Section |5| to compute either the 
minimum or maximum reachability probability. 

Several aspects of the abstraction-refinement implementation merit further 
discussion. In particular, the refinement process presented in Section [5] discusses 
the refinement of a single symbolic state. Because each refinement requires a po- 
tentially expensive numerical solution phase, an efficient scheme to select which 
state (or states) are to be split is essential. In fact, we found it possible to obtain 
very good performance with relatively simple heuristics. In the results presented 
here, we simply refine all states for which the lower and upper bounds differ. 

Our implementation includes several useful optimisations. Firstly, we modify 
the BuildGame algorithm so that it only rebuilds states of a stochastic game that 
have actually been modified during refinement. Secondly, we use the techniques 
described in to re-use numerical results between refinement iterations, re- 
ducing the amount of numerical solution required. 


Experimental results. We evaluate our implementation on 7 large PTA case 
studies from the literature: (i) csma and csma abst, two models of the IEEE 802.3 
CSMA/CD protocol; (ii) firewire and firewire abst, two models of the IEEE 1394 
FireWire root contention protocol; (iii) zeroconf, the Zeroconf network configura- 
tion protocol; and (iv) nrp honest and nrp malicious, two model of Markowitch 
& Roggeman’s non-repudiation protocol. Full details of all these case studies, 
their parameters, and the properties checked are available[] 


3 http: //www.prismmodelchecker.org/files/formats09 / 
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Case study Game-based Backwards Digital clocks Min/Max 
(parameters) verification reachability reachability 
[min / max] Iters] States [Time (s)|| States [Time (s)|| States [Time (s) || probability 
csma 24 10| 6,476 3.9 243 20.7 n/a n/al] 0.143555 
(maz_backoff | 2 8 10| 18,196 8.9 575 77.8 n/a n/a|| 0.005259 
collisions) | 44 10) 34,826 20.5 303) 1443.7 n/a n/a|| 0.076904 
[max 48 10|239,298| 431.4|/time out|/time out n/a n/a|} 1.65e-5 
csma co 0 117 0.2 0 8.7 5240 21.2 .0 
abst 1000 0| 6,392 1.9 366 68.2|| 1,876,105 71.2|| 0.0 
(deadline) |2000 37| 24,173 20.7 722 367.8|| 6,570,692 651.8]| 0.869791 
[min 3000 76| 79,608] 448.0 1,736| 1436.3]}11,780,692 1951.9|| 0.999820 
firewire oo 0 257 0.7 127 26.4 212,268 39.7 .0 
(deadline) 25 0| 1,369 2.0 1,004 839.5||14,089,691 324.6|| 0.5 
. 50 7| 4,215 10.6 3,096} 3149.9]| time out] time out]| 0.78125 
[min 75 34| 10,252 83.4||time out|time out|| mem out|mem out]! 0.931641 
firewire oo 0 10 0.03 0 1.0 776 0.3]] 1.0 
abst 50 7 205 0.25 63 2.4 298,010 14.5] 0.78125 
(deadline) | 100 9| 1,023 1.76 180 3.8 686,008 36.4]| 0.974731 
[min 200 40| 9,059 26.1 640 26.4|| 1,462,010 149.2]| 0.999630 
zeroconf o0 0 26 0.17 19 0.22 357 1.69|| 0.001302 
(deadline) 100 0 132 0.16 15 0.32 8,423 0.93]] 6.52e-4 
150 3 380 0.44 101 0.72 23,888 1.71]} 0.001073 
ae 200 7| 670| 0.73 274 4.77 41,713 2.92|| 0.001222 
nrp co 0 5 0.04 0 0.70 n/a n/a|| 1.0 
honest 40 9 428 1.80 33 5.25 n/a n/a|| 0.612580 
(deadline) 80 39 1,448 3.56 63 6.18 n/a n/al| 0.864915 
[min 100 49| 2,183 5.35 78 6.97 n/a n/a|| 0.920234 
nrp oo 1 351 1.3 62 1.5 n/a n/al] 0.105658 
malicious 5 3 1,663 1.5 75 2.9 n/a n/a|| 0.1 
(deadline) 10 5| 8,080 11.1 408 117.3 n/a n/a|| 0.105444 
[max 20 7| 49,622| 218.1 1,108; 1606.5 n/a n/a|| 0.105657 
Table 1. Performance statistics and comparisons for game-based PTA verification 


We present a comparison of our implementation with the two other exist- 
ing techniques for reachability analysis of PTAs: backwards reachability [I7] and 
digital clocks \15|. For the former, we use the implementation of which uses 
PRISM as a back-end to analyse MDP. For the latter, we use a simple language- 
level translation. We do not consider the MDP-based forwards reachability al- 
gorithm [16[5] since this does compute exact probability values and is thus not 
directly comparable. All experiments were run on a 2GHz PC with 2GB RAM. 
Any run exceeding a time-limit of 1 hour was disregarded. 

Table |1| summarises the experimental results. We give, for each PTA and 
each applicable analysis technique|"] the total time required and the size of the 
probabilistic model constructed. For backwards reachability and digital clocks, 
this model is an MDP; for our approach, it is a stochastic game (we give the 
size of the final game constructed during abstraction-refinement). For backwards 
reachability, the time given includes both generation of an MDP and its solution 
in PRISM; for digital clocks, the value is just the solution time in PRISM. For 
our game-based verification approach, we give the total time for all steps: reach- 
ability graph generation and multiple iterations of game construction, solution 
and analysis. The number of refinement steps required is also shown; in all cases, 
we refine until precise values are obtained (i.e. e=0). Finally, Table [1] also gives 


* The digital clocks approach is not applicable to several of the case studies since the 
PTAs contain zones with strict constraints. 
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the actual reachability probability for each model checking query and whether 
this a minimum or maximum value. 


Analysis of the results. Our game-based approach to PTA verification per- 
forms extremely well. In all cases, it is faster than both backwards reachability 
and digital clocks, often by several orders of magnitude. We are also able to 
analyse PTAs too large to be verified using the digital clocks approach. 

In terms of the size of the probabilistic models generated by the three tech- 
niques, we find that backwards reachability usually yields the smallest state 
spaces. This is because it only considers symbolic states for which the required 
probability is greater than 0. Thanks to the fact that our approach avoids some of 
the complex zone operations required for backwards reachability, we are able to 
consistently outperform it, despite this fact. On PTAs with a very small number 
of clocks (e.g. firewire abst has only 2), the overhead of these complex operations 
is reduced and backwards reachability performs better. By contrast, for PTAs 
with more clocks (firewire has 7 and csma has 5), the opposite is true. 

The reason that our game-based technique outperforms the digital clocks 
approach is that the latter generates models with much larger state spaces, which 
are slow to analyse, even with the efficient symbolic techniques of PRISM. 


7 Conclusions 


We have presented a novel technique for the verification of probabilistic au- 
tomata, based on the use of two-player stochastic games to represent abstractions 
of their semantics. Our approach generates lower and upper bounds for either 
minimum or maximum reachability probabilities and then iteratively refines the 
game to compute the exact values in a finite number of steps. We have imple- 
mented this process and shown that it outperforms existing PTA verification 
techniques on a wide range of large case studies. 

Our approach can easily be extended to compute expected-reward properties 
for the case where rewards are associated with transitions of a PTA. Furthermore, 
we plan to adapt our techniques to compute lower and upper bounds on more 
general classes of rewards properties. Another direction of future work is the 
investigation of improved abstraction-refinement schemes. The simple approach 
adopted in this paper works very well but we anticipate that there is considerable 
scope for improving performance further in this way. Finally, we also plan to 
apply this approach to the verification of real-time properties of software. 


Acknowledgments. The authors are supported in part by EPSRC grants 
EP/D07956X and EP/D076625. 
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